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ABSTRACT * 

1 \ 'To relate the design of open education programs to 

their outcomes, the authors surveyed 72 research studies on the 
effects of open education and examined the methods and findings of 
three previous literature surveys on this topic* The 72 studies were 
divided by an objective statistical test into those showing large 
-effects of open education and those showing small effects. The 
researchers then compared seven features and four aspects of open 
education with six student outcome variables, controlling for nine 
characteristics of the research studies 1 design and methodology. 
Amon$ the open education features were multigrade student groupings, 
open space, individualized instruction, and team teaching; open 
education aspects included the number and extensiveness of open 4 
education features in each study. Outcome variables comprised 
students 1 self-concept, creativity, attitude toward /^School, and 
reading* math, and language achievement. The results showed that the 
size of open education effects varied for different features and* 
outcomes. For instance, for student self-concept, open education * 
programs showing large effects averaged two more features than those 
showing 'small effects, while for student creativity,, programs with 
larger effects had one more feature* (RW) 
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Identifying Features of Effective Open Education Programs 

The identification, of general effects for open education is complicated 
by the fact that open education is not a single, well-defined treatment. Open 
education programs which all share some common philosophical assumptions 
about the nature, development, and learning of children, nonetheless differ 
widely in both the types of features knd number of features of open education 
which are implemented (Giaconia, 1980), Some open education programs emphasize 
open space as the salient feature of a good open education pro'gram: other pro- 
grams emphasize teaching practices and the roie of the child; still other 
programs emphasize some combination of open space and teaching practices. 

These naturally-occurring variations in open education programs would 
not hinder „ef forts to draw general conclusions about the efficacy of open 
education, if a consistent treatment effect could be identified across all 
the empirical studies of open education programs. But ooen education seems 
to produce a vai^iety of effects across all program implementations. Some open 
education programs produce particularly large positive effects for student outcomes 
such as self concept, reading achievement, creativity, locus of control, 
mathematics achievement, and attitude toward school. Yet other open educa- 
tion programs yield large negative effects for tliese same student outcomes 
(Hedges, Giaconia, & Gage, 1981). 

This paper reports on the attempt tcT relate the observed variability 
ia the featur-es of open education Vhich were implemented in different open 
education programs to the observed variability in program effects. 
In order- to achieve this end, this paper suggests a strategy for: (1) ident- 
ifying effective versus less e ffective o p en education programs, by use o f 

an unbiased estimate of effect size rather uhan the statistical significance 



of the reported results: and (2) identifying the distinctive features of effective 
open education programs, by a comparison of the features of open education 
found in the one-third of the studies that /ielded.the largest positive effect 
sizes and the one-third of .the studies that yielded .the largest negative effect , 
sizes. These descriptions of the^f eatures of open education that distinguish 
effective from less effective open education programs can serve as the start- 
ing point for the design of more systematic studies to test the relative 
contribution of each open education program feature to overall program 
effects • 

9 

The Open Education Debate 
The relative merits of open and traditional education have been a 
subject of debate since -the time of Socrates (Broudy & Palmer, 1965). 
Over the years, a plethora of terms has emerged for innovative educational 
programs that all seem to share some common philosophical assumptions and 
observable features of open education/ e.g. , progressive education, informal 
education, free school, open space school, open corridor school, integrated 
day plan, alternative school, and so on. In recent times, anepdotal evidence 
and logical analysis have yielded to empirical research as admissable 
evidence in the open versus traditional education debate. Proponents of 
both open and traditional education have tried to bolster their claims 
about the efficacy of- open or traditional education with systematic empir- 
ical studies of open education programs. Horwitz (1979) identified over 200 
of these empirical studies. 

The persistence of the open education debate, and the large number of 
empirical studies that have been generated by this debate, both suggest the 
importance of " attempts to search for meaningful ways to summarize the empir- 
ical findings, to identify general effects for open education. Three of these 
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recent attempts will be reviewed briefly: a review by Horwitz (1979), a 
^meta-analysis by Peterson (1979), and a meta-arfalysis by Hedges, Giaconia^ and 
Gage (1981). 

T4ie Review by Horwitz 

Horwitz (1979) identified about 200 empirical studies ^that evaluated 
open education programs. Most of these studies involved comparisons of an 
open education program with a traditional education program. Studies were 
included in the review if the educational treatment had either been ex- 
plicitly labeled with the term "open" or if it had been described as having 
characteristics generally ascribed to open eduction, such as flexibility 
of space," student choice of activity, richness of learning materials, 
integration of curriculum areas, and mor6 individual and small-group than 
large-group instruction. 

Nine student outcome variables were reported in this review: academic 
achievement, self concept, attitude toward school, creativity, independence 
and conformity, curiosity, anxiety and adjustment, locus of control, and 
cooperation. 

Horwitz used a "box score" method' to summarize findings across the 
studies. That is, for each^ student outcome variable, he tallied the number 
of studies whose results could be classified as either "open better", 
"traditional better", "mixed results 1 , 1 or "no significant differences/ 1 
Conclusions were drawn abdut the effects of open education on the basis of * 
which category received the most tallies. Horwitz found that" in many instances 
the studies showing no significant differences or mixed results outnumbered tho 
studies showing open better or traditional better.. 

For the student outcome variables academic achievement, self concept,* 
anxiety and "adjustment, and locus of control, the "no significant differences" 



category received the most tallies. For the student outcome variables attitude to- 
ward school, creativity, independence and conformity, curiosity, and cooperation, 
the "open better" category contained the largest number of studies, 

X 

Horwitz concluded, "At this time* the, evidence from evaluation studies 
of the open classroom's effects on children is not sufficiently consistent, 
to warrant an unqualified endorsement of ; that approach to teaching as 
decidedly superior to more traditional methods (p. 83)," 

The Meta-analysis by '^terson 

Peterson (1979) completed a meta-analysis of the studies reviewed 
by Horwitz and other studies that she located. She used Cily those 45 
of th£ studies that contained enough information to permit calculation 
of effect sizes (means and standard deviations of open and traditional 
education groups). Peterson did not retrieve the doctoral dissertations 
reviewed by Horwitz and found that the abstracts qf these dissertations 
often contained too little information to compute effect sizes. Thus 
Peterson's sample of 45 studies was only about 1/4 as large |is Horwitz' s 
original sample, 

Peterson's rationale for undertaking a meta-analysis of a sample of 
the same studies that Horwitz had already reviewed was to eliminate two 
major problems inherent in the box score or vote-counting method that 
Horwitz used. First, the box score procedure maximized* the likelihood 
of Type II errors when the studies were sorted according to the statistical 
significance of the result (rather than the direction of the results regard- 
less of statistical significance). That is, because sample sizes in most 
studies of teaching (including studies of open education) are small and 
because effects for any single teaching variable are expected to be small 
(Gage, 1978, pp. 26-27), use of this procedure increases the probability 



of incorrectly concluding that there was no difference between open and 

traditional education wjien a true difference may have existed. In general, 

0 

. r 

£he box score or vote-counting procedure has a high probability of a 
Type II error when s pie sizes or treatment effects Cor both) are small 
(Hedges & Olkin, 1980) . 

Second, the box score method used by Horwitz provided no indication 
of the magnitude of the open education treatment effect. Glass (1978) 
argued that the box score procedure does not allow the research integrator 
to determine whether a treatment ,f wins by a nose or a walkaway." Glass 
recommended calculating an "effect size" for each study, as a measure of 
the strength of the treatment effect. 

Feterson computed an effect size, using Glass's estimator of effect 
size j for each study, for each student outcome variable. Glass s estimator 
of effect size is given by 



X E ~ X C 



s 

C . * 

where _ sam pi e m ean of the experimental group (.open education group) 

< * * 

C = sample mean of the control group (traditional education group) 
s^ . = standard deviation of the control group 

9 

(In* general,, an effect size can be interpreted directly as the difference t 
between the average performance in the experimental and control groups, 
in z~score units. Alternatively, because z-scores can be translated into 
percentile ranks, an effect size can be interpreted as the percentile rank 
corresponding to the performance of the average person in the experimental 
group. For example, an effect size *of +.50 indicates that the average 
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performance in the experimental group is one-half of* a standard deviation 

higher than the average performance in the control group. Similarly, an 

effect size of +.50 indicates that while the performance of the average person 

in the control group. is at the 5T)th percentile, the performance t of the average 

0 

person in the experimental group is at the 69th percentile.) 

Peterson' then averaged the .effect .sizes for all studies, for each of the 
student outcome variables: composite achievement, mathematics achievement, 
reading achievement, creativity, pfcoblem-solving, self concept, attitude 
toward school, attitude toward teacher, curiosity, locus of control, 
anxiety, and independence* Peterson noted that in the case of most of * 
the student outcomes,* effect sizes were quite small, indicating little 
adavantage for either o^en or traditional education. She based conclusions 
about the effects of open education on both the direction and magnitude 
of the average effect sizes; a positive effect size meant average performance 
was higher in the open education groups, while a negative effect size • 
indicated that average performance was higher in the traditional education c 
groups. 

Average ef-.'C sizes for jnathema tics achievement, reading achievement, 
and composite achievement were negative and showed about 1/10 of a standard 

deviation (or slight) adavantage for the traditiona l educa tiojL-groups*- - 

Average effect sizes for locus of control and anxiety were near zero, * 
showing no advantage for either open or traditional education. 

For the student outcomes creativity, attitude toward school, and curios- 
ity, average effect sizes were positive and indicated an advantage for open 
education students between 1/10 and* 1/5 of a standard deviation. 

Independence and attitude toward teacher yielded relatively larger. effect 

c 

sizes that* showed an advantage for the open education Students of between 
1/3 and i/2 of a standard deviation. 



Peterson concluded (about the main effects of open education), "•••although 
a more direct or traditional approach appears, to be better than a more open 
^pproach for increasing students 1 achievement, an open approach appears to 
Jbe better than a more direct approach for increasing students' creativity, 



/ 

/ 
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i independence, -curiosity, and favorable attitudes toward school and learning 
(p. 67)7 * 

< v • 

The" Meta-analysis by Hedges, Giaconia, and Gage 

Hedges, Giaconi%, and Gage (1981) undertook yet another meta-analysis 
of research on open education. This meta-analysis was designed to improve 
upon the 'meta-analysis by Peterson in at least three ^ays . First, doctoral 
dissertations were retrieved and used in the analysis, Peterson had excluded 
these from her meta-analysis. These dissertations numbered about 90 and 
constituted over half of the total sample of Studies used by Hedges and his 
co-workers. 

> 

* Second, effete t sizes were computed using an unbiased estimator of 
effect size, for which Hedges (1980) had developed the statistical theory. , 
This estimator of effect size is given by 

= c 

q m - » 

pooled 

where X^. == sample mean of the experimental group (open education group) 

== sample mean of the control* group (traditional education group) 
Spooled S P°°^ e< ^ estimate of the standard deviation 

* C - constant for m degrees of freedom, where m s n_ + n_ - 2 
m h \j 

The exact formula for is given by Hedges (1981), but a good approximation 
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is^ given by . ' ' / 



*1 - 

4m - V 



where m = n_ +. n„ - 2* . " 

b « c * ✓ 

The third way in which the meta-analysis by Hedges, and his co-workers'* 

was designed to impreve upon the meta-analysis by Peterson was by taking * ~ 

into account various characteristics. of each f study. These characteristics of each 

study included subject characteristics (grade, SES, sex, ethnicity, entering 

^ability); experimental design characteristics (method of determining opfenrie'ss; 

method of assigning students to open and traditional education "programs, 

duration of the treatment, unit of matching, and unit of analysis).; experimenter . 

» 

characteristics (rating of t the predisposition of the investigatott^o favor 
open or traditional, clarity of the research report); and experimental context 
variables (type of publicapion, year of publication) . This information \<ra$ * ' 
used to test multifacef interaction models for the series of studies on all 
possible comparisons of open and traditional education, i.e., 'to look for 
differences in the magnitude of effect sizes for various configurations of these 
study characteristics, e.g., differences in effect sizes for experiments fl 
versus correlational studies, for younger students versus older student's, 
for studies with longer treatments, and those with shorter treatments, and so on. 

*' Hedges and his co-workers used a sample of about 153 studies that was 
similar to the sample of steadies reviewed by Horwitz. A few studies that 
had been published after Horwitz f s bibliography was prepared were added. 
Several studies that Horwitz had reviewed were excluded because a complete 
report could not be obtained, because the study did not compare open education 
to any other group, or because it appeared that the open education group 
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was not receiving* any open education treatment {i.e., the treatment did not' include 

any elements of open instruction* or open space). . 

Hedges' (1980) unbiased estimator of effect size wa& used, to compute an 

effect size for each comparison *of open and traditional education, in each 

study, for each student* outcome variable ^eparately. Tfiat is, rather -than 

computing. one effect size pef* study for each student outcome, Hedges and'liis co-worker 

computed separate effect sizes for each independent comparison, if enough information 
to do so, has been provided, i.e. if means and standard deviations were tabled or 
could be obtained algebraically* For example, a study might report means and ^tandard 
deviations for dpen-traditflonal comparisons at each of three grades • Three separate • 
$nd independent sizes were computed, one for each grade. Tf additional data were . i 
reported such at> in breakdowns by sex or SES withiit a study, the effect size estimates 
were competed but not used in subsequent analyses* When a s^udy did not provide 
enough information to permit calculation of an effect size, only th6 direction of 
the difference was recorded (favors open, favors traditional, or favors neither). 

• * 
Hedges and* his co-workers identified 38 student outcome variables 

and reported average effect sizes for 16 of these: achievement motivation, 

* * * 

adjustment, anxiety, attitude toward school, attitude toward teacher, coopera- 
tiveness, creativity, curiosity, general mental ability, independence and self- 
-reliance, Iqcus of control, self concept, language skills achievement, math- 
ematics achievement, reading achievement, and miscellaneous achievement. 

The average effect sizes for each of these student outcomes were based 
on the unweighted average of the' comparisons across studies. (Each study 
may have contributed a different number of comparisons to the average.) 

Hedges and his co-workers also reported the percentage of studies for which 

/ * * 

the direction of the effect f§vored open, favored traditional ,jor favored 

neither 

The average effect sizes for the student outcomes adjustment, attitude toward 



school, attitude toward teacher, curiosity, and general mental ability, 
were all positive and showed an advantage for the open* education group 
of about 1/5 cf a standard deviation. The direction of the effect for the 
majority of studies also favored open education for these student outcomes. 

The average effect sizes were positive, but near zero, for the student outcomes 
locus of control, self concept, anxiety, and miscellaneous achievement. The 
direction of the effect for the majority of the studies favored traditional 
education for locus of control, anxiety, and miscellaneous achievement. For 
-self concept, the direction of effect for the majority of studies favored 
open education. 

For cooperativeness, creativity, and independence, average effect sizes 
were positive and indicated an advantage for open education of between 1/4 
and 1/3 of a standard deviation. The majority of the studies showed a direction 
of effect that favored open education for all of these student outcomes. 

The average effect sizes for language achievement, mathematics achievement, 
and reading achievement were negative, but near ,?ero, indicating no particular 
adavantage for either open or traditional education. The direction of effect for 
"the majority of studies,* however, favored traditional education. 

.Hedges and his co-workers concluded that some of the claims of pro- 
ponents of open education had been supported, but that open education did 
not produce consistent effect across the different student^ outcomes . Open 

« 4 

education macte its strongest showing for the student outcomes creativity, 
coooerativeness, independence and self reliance, attitude toward teacher, 
curiosity, attitude toward* school, and adjustment. Open education made its 
weakest showing for reading, mathematics, and language ^achievement . 
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Summary of the Reviews of Open Education 

Table 1 summarizes the results reported in the three reviews of open 
education by Horwitz, Peterson, and Hedges and his co-workers. The table shows 
no major discrepencies In the three reviews for those student outcomes 
which all three reported. Student outcomes for which Peterson 

and* Hedges both reported average effect sizes near zero (e.g. anxiety and 
locus of control) were also outcomes in which Horwitz's "ri^ significant diff- 
erences" category contained the most studies. Student outcomes for which both 
Peterson and Hedges and his co-workers reported relatively larger effect sizes 
(e.g. ^creativity, independence, attitude toward school, curiosity), were also 
outcomes in which Horwitz's "open better" category contained the most studies. 

Two major conclusions seem warranted from the results reported in 
Table 1. First, in general, open education is only somewhat more effective 

o 

f 

that traditional education and for only some student outcomes. Traditional 
education is only slightly more effective than open education for the 
traditional academic achievement measures. For many student outcomes, there 
are near zero differences, between open and traditional education. 

Second, these general conclusions about the effectiveness of open 
education must be tempered by the fact that the variability of the effects 



of open education programs is often quite high. The ranges of effect size 
reported by^ Peterson for each student -outcome were quite large. For example 
mathematics, achievement yielded an average effefct size of -.14, but the 
rar^ge Was -1.01 to +.£l". Similarly, £he standard deviations of effect size 
reported by Hedges and his ^ co-workers tiere large. For example, mathematics 
^achievement- yielded an average effect size near zejro (-.034), but the 
'standard .deviation toas .383. Thus, while thfe av'erage effect sdzes across 
studies were'in most cases quite small, some studies, produced particularly 
large positive or ^particularly large negative effect sizes. It is these. 
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Summary of the- Results Reported 
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extreme groups of studies which this paper examines. 

Rationale for this Study 
The wide variability in the numbers and types of features of open educa- 
tion which are implemented in open education programs has been documented by both 
researchers and reviewers of open education (cf. Horwitz, 1979)- Similarly, the 
wide variability in the sizes of effects produced by different open education 
programs has been shown by the ranges and standard deviations in Table 1. Further, 
Hedges (1981) developed a statistical test of the homogeneity of effect sizes. This 



procedure tests whether the obtained effect sizes could have arisen by chance in 
.samples from populations with the same underlying effect size. This test of 
homogeneity, when applied to the effect sizes computed in the meta-analysis by 
Hedges and his cq-workers, showed that for; every student outcome variable the 
variability in effect sizes was greater than. could be accounted for by error or 
random sampling variability- Some, factor or factors other than chance were con- 
tributing to the variability in effect sizes across studies. 

This sjzudy examined the relation of the observed variability in effect 
sizes to the observed variability in the numbers and types of open education 
features implemented and to study design characteristics. 

Methods 

The 153 studies of open education which were used in the meta-analysis 
by Hedges and his co-workers served as the dA^ base- For each of the 16 
student outcomes, studies were sorted into one of three categories on the 
basis of the magnitude and direction of the unbiased estimate of effect 
size for the study. (When more than one effect size had been computed 
per study for each student outcome, the median effect size was used as the basis 
for classifying the study,) 

"Large effect 11 studies were the one-third of the studies with, the 



largest positive eif feet -sizes'. "Small effect" studies were the one-third of 
the studies with .the smallest effect sizes, including those less than -zero. 
n Medium effect" studies were the one-third of the studies remaining after the 
large effect and small effect studies had been identified. Only large and small 
effect studies were used in the subsequent analyses. ' 

Since statistical significance., depends on sample size, a high degree of 
statistical significance is not equivalent to a large effect magnitude. Indeed, 
the reason for effect size indices stems from ^the need for an index of effect 
magnitude that is independent of sample size. Therefore we sorted the studies 
into large effect (effective) versus small effect (less effective) categories 
on the basis of effect size rather than statistical significance of the repotted 
result. Sorting the studies into large effect and small effect categories for 
<feach 'student outcome variable separately was more theoretically justifiable than 
trying to identify generally (across student outcomes) effective versus generally 
ineffective open education programs. One property of effect sizes is that they 
are homogeneous with respect to linear transformation (Hedges, 1980). Therefore 
two variables *tti#t are perfectly correlated will have the same effect size. But 
because different student outcomes are not linear transformations of each other 
(e.g. measures of reading achievement are not linear transformations of measures 
of mathematics achievement, i.e., they are not perfectly correlated), combining 
effect sizes across student outcome variables is inadvisable. 

A total of 72 different studies were used in this study. Table A in the 
appendix lists the studies and how they were classified (large, small, or medium 
effect) for each student outcome. 

Descriptions of Student Variables . - 

The student outcome variables used -in this study were those for which 
there were at least seven studies in each of the large effect and small effect 
categories. These outcome variables were self concept, creativity, attitude 
toward school, reading achievement, mathematics achievement, and language 
achievement. 

Table 2 summarizes the number of large effect and small effect studies 
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Table 2 

Average Effect Sizes for All Studies, Large Effect Studies, and Small Effect Studies 

All Comparisons 

Across All Studies Large Effect Studies Small Effect Studies 

Student Outcome M SD n a ■ M - SD n b M SD n 



3 

Self Concept .056 .420 84 .641 .423 16 -.347 .249 ' 16 

Creativity .302 .403 23 „ .724 .187 7 -.169 .234 7 

Favorable Attitude _ \ * " *" ' 

Toward School .166 .447 68 .730 .279 13 -.329 .253 13 



Reading 
Achievement 

Mathematics 
Achievement 

Language 
Achievement 



\ 



-.038 .362 73 .344 .3^5 17 -.498 .136 17 

-.034 .383 62 .387 .204 14 " -.616 .259 14 

-.053 .581 32 .494 .434 , 8 -.590 .147 8 

\ . . - 



r 2 ^* 3 n equals number of comparisons; some studies yielded more than one comparison 
* ' per student outcome, e.g., separate effect sizes were computed for each grade 

level within a study. ► 

b n equals number of .studies; one effect size per study (per student outcome) was 
used in computing means and standard deviations. 

15 ■ ' ' ' 49 
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that were compared for each off the six "student outcome variables. It also 
lists the mean effect sizes and standard deviations for each student outcome, 
for all the studies, for the large effect studies, and for the small effect 
studies. The average effect sizes for the large effect studies are all 
much higher than the average effect sizes for both the small effect studies 
and all studies* Similarly, the average effect sizes are all much lower 

• for the small effect studies than the average effect sizes for either all 
studies-or large- effect studies. Thus, the large effect" and " small effect 
studies do seem to represent* extreme groups in terms of average effect size* 

Table B in the appendix lists the measures used in the large and small 

. effect' studies for each student outcome. The numbers and types of different 
measures used for each student outcome provide some information about the nature of the 

it 

student outcome construct as it was actually measured. 

Self concept was broadly defined to include self-appraisal, self- 
security, or self -acceptance in both acad£Tiic and general life situations. 
Fifteen different measures of self concept were reported in the 32 large 
effect and small effect studies. Most of these measures of self concept 

* 

were group-administered, < student self report inventories. The measures 
differed primarily in terms of the situational specificity of self concept, 
e.g., general self concept versus self concept in academic situations 
versus physical self concept. There are reasons to suspect that 
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construct interpretations from such a large set of different measures should 
be made cautiously. Shavelson, Hubner, and* Stanton (L976) , in a review of 
studies of self concept, argued that there is a general lack of an agreed- 

. - * 

upon definition of self concept, a lack of ; adequate validation of inter- 
pretations of self concept measures, and a lack of empirical data on the 
equivalence of the self concept measures currently being used. 

Problems may also arise in the interpretation of scores on some 
measures .of academic self concept, if the actual ability level of -he 
student is not taken into account. "Yes" responses to items like 



.Jt Ml 



,: I think my grades are poor', 1 :t I think I am poor in arithmetic," and "Host 
of my grades are lower than other kids' grades" may be a realistic self-evaluation 
on the part of a low-ability student and poor self* evaluation or excessive 
self-criticism on the part of a high ability student ,T A desirable student 
outcome may be for all students to feel good about themselves no matter 
what their ability levels are. But this should be distinguished from unrealistic 
self evaluations of ability. Thus higher scores on self concept measures; ^ 
of this sort may not always represent a desired student outcome. 

Creativity was defined to include the fluency, flexibility, originality, 
and elaboration dimensions of behavior, in both the verbal and figural 
domains. Six different measures of creativity were used by the 14 large 
effect find small ^effect studies. Most of these studies reported scores 

s 

for the Torrance Tests of Creative Thinking, Most of the measures of creativity 
that were used^ Required the student to produce either ideas, figures, or drawings 

s . 

in response to a verbal or figural stimulus. Only one measure (Pennsylvania 
Assessment of Creative Tendency") was a self-report inventory that measured 
attitudinal factors which support creative behavior- Thus the measures 
reported for creativity are fairly homogeneous with respect to the construct 

c 

that they purport to measure, 

21 \ 
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/Favorable attitude toward school was defined to include favorability 
toward, interest in, or appreciation of various aspects of school life, 
such as learning, peer relations, physical "environment, structuring of 

9 

. tasks, teacher's role, school and classroom policies, and so on. Eighteen 
different measures were reported in the 26 large and small effect studies. 
Most of these measures of attitude toward school were student self-report 
inventories, and many were instruments developed by the investigator for 
his or heir particular study of open education. 

The measures of attitude toward school differed widely in the extent 
* to which they emphasized the various aspects of school life (e.g. physical 
environment, teacher relations), and in the breadth of coverage of the 
aspects of school life. 

One crucial problem with a few of these measures of attitude toward 
school was that they seemed more like measures of implementation of an open 
education program than measures of the desired student outcome of favorability 
toward school. For example, one measure that used a semantic differential 
approach, asked students to indicate on a seven-point scale the extent to 
which they generally felt the anchored adjective described their feelings 
toward the classroom, 6ne item used "teacher-centered" and "student-centered 11 
as anchor points. • But by definition, the open education program in this study 
was student-centered and such an item may merely reflect whether this aspect of the 
program was implemented, and not students 1 favorable attitudes toward school. 

Thus the measures used for attitude toward school are fairly hetero- 
geneous and it is not clear that they are all measuring the same construct. 

Reading achievement represented a summary measure of all reading 

subskills, such as reading comprehension and vocabulary. Twelve different 
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measures of reading achievement were reported in the 34 large effect and 

small effect studies. 

Mathematics achievement represented a summary measure of mathematics 

subskills such as computation, concepts, reasoning, and problem-solving. 

Ten different measures of methematics achievement were used in the 28 , 

large effect and small ef-fect studies. 

Language achievement represented a summary measure of language sub- 
^skills, such as spelling, usage, and capitalization and punctuation. Eight 

different measures of language achievement were reported in the 16 large 

effect and small effect studies. 

All of the achievement measures for reading-^ mathematics, and language 

are quite homogeneous.. Most of these measures were subtests of standardized 

achievemeat batteries, for which extensive reliability and validity data. had 
-beetle gathered. Thus the interpretation of effects across the different 

reading, mathematics, and language measures is probably less ambiguous than the 

interpretation of effects across the different self concept and attitude toward 

school measures. 

Description of the Design Characteristics of Studies 

Nine different design characeristics on which the large effect and 
small effect studies Could differ were identified. These characteristics 
included some of the subject variables, experimental design variables, experi- 
menter "variables, and context variables on which Hedges and his co-workers 
collected information for their meta-analysis. Following are brief descriptions 
of the characteristics and categories on which tt>e large effect and small 
effect studies were compared. 

(1) Design of the study <p- Two categories were included * correlational 
and "experimental or quasi-experimental. It was necessary to combine 
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experiments and quasi-experiments into one category because too few 
studies were tfcue experiments. The pyrpose of including this character- 
istic was to look for systematic differences in the extent -to which * 
large effect and small effect studies allowed causal inferences to be 
made about, the results reported in each study. 

(2) Type of publication — Four categories were included: dissertation, 

<- • 

journal article, book or monograph, and ERIC document. Dissertations 

• • < 

4 

typically provided more detail about the open .-education treatment 

while journal articles were typically reports of better-designed studies. 

(3) ^ Method of determining open education treatment — Three categories 
.were included: observation or questionnaire, o^en space (architecture) , 
.and judgment of the investigator '(s") # T'uis characteristic was thought 

to be an index of treatment fidelity and strength, i.e., whether 
implementation of the open education program was actually measured 
(observation or questionnaire) or ^trstvassumed (judgment of investigator) 
and whether the open education treatment was based on open space 
alone or on teaching practices (questionnaire and judgment). * 

(4) Grade level of students — Four categories were included: 
kindergarten to grade2, graHes 3 and A, grades 5^ and 6, grade7 and above. 
This characteristic was studied because some researchers have questi^ped 
whether the assumptions underlying open education are eaually true for 
children of different ages (cf. Traub, Weiss, Fisher, & Musella, 19/2). 
This characteristics also provided an indirect index of the strength of the 
open education treatment. Younger students in open education programs 
are~mofe likely than older students to have experienced only open education 
older students in open education programs are more likely than younger 
students to have experienced both open and traditional education programs. 



(5) Degree of comparability df open and traditional students — Three 
categories were u£>ed;-,high, meja&im, and low. 'Coding this characteristic^ 4 
involved a judgment by the 'reader of the study.' , Itif ormation such as 
mention of random assignment of students; matching of students, glasses, 
teachers, or schools ; * • ■ 

was used in evaluating the degree of comparability df students in the open 
and traditional education groups. 

In making this judgment, the reader of the study also looked at the 
amounts of open or traditional education to which each group had been exposed 
A common problem in many of th£ studies was the* comparison jbf traditional ** 
education students who se> whole. educational experience -had been in traditional 
education programs with open education students who were new to open * *" 

v : 

education and who had experienced unknown prior amounts of traditional 

*>'■" r . 

education. * 

i 

• * 
This design characteristic was included because the comparability 

of students in the experimental (open* education) and control groups 

bears directly on the validity of causal inferences that can be made from 

the 'results of a study. 

C6) Country in which ' the study was implemented — : "Two categories were 
included: USA and Britain, Canada or Australia. This characteristic was 
studied because of qualitative differences that may exist between open 
education programs in the USA and Britain. Britain has a longer history 

y 

of open education and the open education programs that are implemented 

there tend to be more homogeneous with respect to the model of open education 

on which they are based than the open education programs implemented in 

the USA. ' 

(7) Duration of the open education treatment — ~ Four categories were used: 
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yless than one, year, one school year, between one and two years, and 
more chan two years. -This characteristic was included as an index of 
the strength of the open education treatment; longer open education 
treatments were expected to be related to larger effects. 

(8) Quality of the study ~ Three categories were included: very good,- 
moderately good, and poor/ The reader of the study made a suramary- 
judgment, about the quality or interpretability of ea'ch study, on the basis 
of information such as adequacy of the experimental design, sampling 
procedures, descriptions of the open and traditional education treat- 
ments, explicitness of the research question(s) ,. appropriateness of the 
procedures, used to test the research hypotheses", and quality of the 

\ 

measures used for the student outcome variables. 

This characteristic provided a global index of the ''trustworthiness" 
of the results of the study. Inter-reader agreement *for this character- 
istic was 87%. This percentage of agreement wa»s computed for the 72 y 
studies reviewed for this.study and which had been coded previously iji the 
meta-analysis by, Hedges and his -co-workers . 

(9) ■ Commitment of .the investigator — Three categories were used: 
favors- open education, neutral, favors traditional education. The reader 
of the study made a judgment about 1 the commitment of the investigator (s) 
largely on the basis of the tone of the research report, e.g., from 
comments in the introduction to the report that suggested that the ^ 
study was designed to "support" versus "test" versus "challenge" 

the claims of open education. * 

The commitment of the investigator* could be related to ?^rge or 

t ✓ 

, small effects for at least two different reasons. First, a strong 
• commitment to open education could have resulted in experimenter biases 
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of the sort that produce artifactually large effects. Or, second, *j 
a strong commitment to open education could indicate that the investigator 
was more informed about the types and features of open education programs 
(than the investigator with a commitment to traditional education) and was 
more likely to implement a sound, extensive open education program that 
would produce- large effects. 

Intern-reader agreement for* rating this characteristic was 81% for 
the 72 studies reviewed in this study. 

The percentages of large and small effect studies falling into each of the 

designated categories, for each of these nine design characteristics, 

were determined. The chi-square statistic or Fisher's Exact Test was computed 
♦ 

for each characteristic,- in order to determine the relationship between the 
classification of a study as large effect or small effect and each of the 
design characteristics. 

Descriptions of the Features of Open Education Programs v 

The careful identification of a complete and representative^ eatures . 
of open education programs on which to compare large effect and small effect 
'studies was crucial to this study. Two major decisions had to be made: y 9 

Whij^h features of open education to include in the analysis and how to code 
information about these features. 

The first decision involved a tradeoff between compiling an exhaustive, 
detailed* list of features that would fully capture all the nuances of different 
'open' education programs and the practical constraint that most studies 
included too little information about the open education treatment to conduct 
this fine-grained analysis. 

The features of oper\ education that* were used in this study to compare 
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large effect and small effect studies were based partly .on the general 
categories proposed by Traub, Weiss, Fisher, and Musella (1972), partly on 
the categories described by Walbferg and Thomas v (1972) , and largely on 
general impressions gathered in the course of reading the 153 studies re- 
viewed in the meta-analysis by Hedges, Giaconia, and Gage (1981). 

Traub and his co-workers developed a teacher questionnaire (Dimensions 
of Schooling) which categorized several features that were found %o distinguish 
open education programs from traditional education programs. Two criteria" were 
used by Traub and* his co-workers in identifying these features of open 
education programs. First, the feature could not contradict any of the assump- 
tions about the way children behave, develop, and learn that Barth (1969) 
had identified as central to open education. Second, the feature had to have 
# two or more program manifestations that could be easily ranked in degree of 
openness, *• * 

Ten dimensipns of schooling were proposed by Traub and his co-workers: 
CI) Setting Instructional Objectives -- process by which instructional 
obj-ectives are set, i.e., participation , of students in this process. 

(2) Materials and Activities — diversity of material, activities, 
and media* 

(3) Physical Environment — flexible use of space and furnishings. 

(4) Structure for Decision-Making — student choice in assignment 
to teachers* 

r ^ 

(5) Time Scheduling — no fixed timetables, time for independent 
study, # uns true tured time, no attendance requirements. 

(6) Individualization of Learning — small group instruction, 
individually-^aced instruction, student choice in method of learning, 

(7) Composition of Classes — multi-age grouping of students. 
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(8) Role of Teacher — teacher as resource person, teacher as 
diagnoser of student problems and progress, teacher guides but does not 
force students, teacher wgtrks with individuals or small groups' of 
students. 

(9) Student Evaluation — little or no use of conventional tests, 
purpose of evaluation is to direct student learning, student also 
provides self-evaluations, continuous evaluation, use of observations, 
work^ samples , and anecdotal evidence. 

(10) Student Control role of student in rule-formulation and 
rule-enforcement • 

- — Walberg and Thomas (1972) identified eight themes of, open education 
which they used as the starting point fojr the development of a 50-item 
open education observation scale and a parallel teacher questionnaire. 
The eight themes proposed by Walberg and Thomas were based largely on the 
ten themes that Bussis and Chittenden had arrived «at from their interviews 

with open education teachers. These eight themes and saraple indicators 

»> 

include: - 

(1) Provisioning for Learning — diversity of manipulative materials, 
freedom of movement for students, students group themselves for instruction. 

(2) Humaneness, Respect, Openness, and Warmth -- % environment includes 
materials developed by students, students' activities and ideas are reflected 
in the classroom. 

(3) Diagnosis of Learning Events — test results not used to group 
students, tests used to find out what student knows. 

(4) Instruction, Guidance, and Extension of Learning — individualized 
' instruction, small group instruction, little" use of curriculum guides 

or textbooks • 

2d 
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(5) Evaluation of Diagnosti- Information — teacher writes individual 
histories of each student's development, tests not used to compare 
students, teacher collects samples of student's work, evaluation used 

to guide instruction. 

(6) Seeking Opportunities for Professional Growth- — use of teacher 
aides, teacher relies on colleagues. 

C7) Self -Perception of Teacher — teacher tries to keep all students 
in view in order to make sure* they are doing what they are supposed 1:0. 
C8) Assumptions about Children and Learning — student inv&lveaent, 
warm and supportive emotional climate, clear set of rules and regulations. 

Seven general features of open education were identified for use in this 
study: open space, materials to manipulate, multi-age grouping of students, 
individualized instruction, role of the child in learning', team teachiijjg, 
and diagribStic evaluation. *In addition, four other aspects of the open 
education programs were evaluated: the number of the features of open education 
that were implemented, the extensiveness/of the open education treatment, 
the specificity of the theoretical mod£L of open education on which the program 
was based, and the emphasis of the jptodel of open education on which the 
program was based. 

\ 

sspects of open education programs. The table also lists keywords or descrip- 

tive statements reported in some of the studies that are examples of each 

I 

of the seven features. r 

I 

Table A sliows the correspondences among the features of open education 

identified for ;this study, the dimensions of schooling described by Traub 

i 

and his co-work!ers, and the open education themes reported by Walberg and Thomas. 
The table shows' some evidence of convergence among the three sets of features, 
although some of the features differ in specificity and some of the categories 



j 

Table 3 gives the definitions of each of these seven features and four 



do not overlap perfect 
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Table 3 

Descriptions of the Features of Open Education on which Large Effect and 
Small Effect Studies Were Compared 

Feature: OPEN SPACE 



Definition: Physical environment of the classroom involving flexible 
use of space and furnishings 



Indicators and descriptive statements: 

-open avek classroom - ~ v 
-open space architecture 
-flexible school architecture 
-open instructional area 
-activity centers 
*-flu.id space 
^-decentralized classroom 



-pod facility school 
-open plant facility 
-no interior walls or movable walls 
-school without walls 
-flexible seating arrangements 
-physically unstructured 



Feature: MATERIALS TO MANIPULATE 

* Definition: Presence of diverse set of materials to stimulate student 
exploration and learning. 



Indicators and descriptive statements: 

-seiisory materials 
-exploration and discovery- 
oriented materials 

- use of natural materials 

- rich material environment 
-alternative modalities for learning 

Feature: MULTI-AGE GROUPING OF STUDENTS 



-diversity of materials 
-abundance of instructional aids 

-tactile confrontation with 
manipulative materials 
-real world materials 



Definition: Grouping students for instruction in which grade labels are 
not applied; two or more grades may be housed in the same area. 



Indicators and descriptive statements: 

-family grouping 
-nongraded school 
-heterogeneous age grouping 
-children from different grades 
work together in same classroom 



-ungraded classrooms 
-vertical grouping 
-continuous progress education 



ERIC 
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Table 3 (continued) . ' 

I 

Feature: INDIVIDUALIZED INSTRUCTION 

Definition: Instruction based on the individual needs and abilities 
of each student; individualization of rate of, methods, and materials 
for learning; small group as opposed to large group instruction. 



Indicators and descriptive statements: 

-individualized instruction 
-individualized 'approach 
-individualized work 
-environment responsive to 

individual learner needs I 
-individualizing the curriculum 

Feature: ROLE »0F CHILD IN ^LEARNING 



-individualized goal setting 
-learning in accord with their own 

rate and style 
-small/group or individual instruction 



Definition: Child is active in guiding her own learning; child 
actively chooses materials, Method's, and pac&_of s learning; role of" 
teacher as resource ♦person; less te^cher-c^npereia instruction and 
more ?tudent-centered iristruction. 

Indicators and descriptive stajtementsj 

-voluntary action on .the part of the qhild 
-active agent in his own learning process 
-self-motivated learning 
-student initiates activities * 

-active participant rather* than recipient of commands 
-trust in the student's ability to choose his own learning experiences 
\ -child-centered environment 

-child's freedom and responsibility for his learning and development 
-democratic learning- "atmosphere 
^student sets rate of learning 

-high degree of child contribution to the learning environment 
-teacher as resource person f 
-teacher is authoritative, not authoritarian 

Feature: TEAM TEACHING 

Definition: The sharing in planning and conducting instruction 
offered to the same group of students by two or more teachers: 
use of parents as teaching aides. 

Indicators and^ descriptive statements : 

-team teaching organization 
-team teaching units 

-teachers work together in teems with a team leader 
-large spaces with two or more teachers 
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Table 3 (continued) 



Feature: DIAGNOSTIC EVALUATION 

Definition: Purpose-of evaluation is td guide instruction; little or no 
- use of conventional tests , but extensive use of work samples, observation, 
and written histories of the -student. 

Indicators and descriptive statements: 

-charting of progress toward specific individual goals 
*> -evaluation used to facilitate arid guide learning 

-child's performance not compared to that of other children 
-teacher's record-keeping combines constant jotting in class and 

thoughtful writing about each child 
-less standardized concept of student progress * 
-non-graded approach to. evaluate student's performance 

Aspect : NUMBER OF FEATURES OF OPEN EDUCATION 

Definition: Measure of the breadth of the open education program; 
•the sum of the number of the seven preceding features (open space, 
materials to manipulate, multi-age. grouping of students, individualized 
instruction, role of child in learning, and team teaching) for which 
there was some evidence of implementation in the open education program* 

Aspect: EXTENSIVENESS OF OPEN EDUCATION TREATMENT 

t> 

Definition: Measure of the depth of the open education program; 
three levels: high, medium, low; determined by a judgment of the 
reader of the study on the basis of the descriptions of the open 
education treatment which the investigator provided, 

. Aspect: SPECIFICITY OF THEORY OF OPEN EDUCATION 

Definition: Measure of the extent to which the open education program 
was based on a well-specified model of open education or on a hodge- 
podge of ideas about open education; three levels: high medium, low; 
a judgment by the reader of the study on the basis of comments made 
by the investigator in introducing her study. 

Aspect: EMPHASIS OF THEORY OF OPEN EDUCATION 

Definition: Measure of the extent to wh£ch the open education 
program emphasized open space, open teaching practices \ or both 
open space and teaching practices; a judgment by the reader of the 
-study made on the basis of descriptions of the open education 
program provided by the investigator. 




Table 4 



Comparison- of the 'Categories of Open Education Features Propos^ in Three Studies 



Giaconia and Hedges (1981) 
Open Education Features 



Traub, Weiss, Fisher, and 
Musella (1972) 

Dimensions of Schooling 



Open Space 

Materials to Manipulate 



Multi-age Grouping of 
Students 



Individualized Instruction 



Role of Child in Learning 



Walberg and Thomas (1972) 
Open Education Themes 



Physical Environment 
Materials and Activities 



Composition of Classes; 
Structure for Decision- 
Making 

Individualization of Learning 



Student Control; Setting 
Instructional Objectives; 
Role of Teacher 



Provisioning for Learning*; 
Humaneness, Respect, Openness and 
Warmth 

Provisioning for Learn:(ng 



Instruction, Guidance, and Exten- 
sion of Learning 

Provisioning for Learning; 
Humaneness, Respect, Openness, 
and Warmth 



Team Teaching 
Diagnostic Evaluation 



Student Evaluation 



Time Scheduling 



Diagnosis of Learning Events; 
Evaluation of Diagnostic Information— 



Seeking Opportunities for Professional 
Growth 



Self Perception of Teacher 



34 



° 31 . 

A second decision, after choosing which features of open education on^ 
which to compare large effect and small effect studies, was how to code 
information in the 72 studies; The choice was between coding only the presence 
or absence of a feature in each open education program and coding qualitative 
aspects of each feature. The former approach was chosen because most of 
the studies contained too little information for a finer-grained analysis 
and because the number of studies we were examining for any one student^ outcome 
variable was small. 

The percentages of large effect and small effect studies for which 
each feature of open education was present or absent were determined* 
A chi-squate test or Fisher's Exact Test was computed for each feature, 
in order to determine the relationship "between the classification of a study 
as large effect or small effect and the presence or absence of an o^en 
education program feature. A chi-square statistic was computed when the 
total number' of large effect and small effect studies was greater than 30: 
Fisher's Exact Probability was computed when the total number of studies 
was less than 30.* 

Results and Discussior 

Tables j and 6 report the percentage of large effect studies and the 
percentage of small effect studies falling into each of the different 
categories for each of the nine design characteristics. The values of 
X 2 or Fisher's Exact Probability are also given. Table 5 reports this 
information for the student outcome variables: self concept, creativity, 
and attitude toward school. Table 6 reports this information for the 
student outcome variables: reading achievement, mathematics achievement, and . 
language achievement. 

The X 2 tests reported for each design characteristic are not independent 



Table 5 

.Comparison of Selected Characteristics "of' the Studies Yielding Large Effects and Small Effects for the Student Outcomes: 
Self Concept, Creativity, and Favorable Attitude Toward School 



, Characteristic of 
the Study 



X of 
Large 
Effect 
Studies 



SELF CONCEPT 

% of 
Small 
Effect 
Studies 



Fisher's 
Exact 
Test 
or X 2 



CREATIVITY 



FAVORABLE ATTITUDE TOWARD SCHOOL 



% of 
Large 
Effect 
Studies 



Z of 
Small 
Effect 
Studies 



Fisher's 
Exact 
Test 
or X 2 



X of 
Large 
Effect 
Studies 



% of 
Small 
Effect 
Studies 



Fisher's 
Exact 
Test 
or X 2 



Design of the Study 

Correlational 

Experimental or Quasi- 
experU.ontal . 

Tync of Publication 
Dissertation 
Journal Artlcl * 
Book or Monograph 
ERIC Document 



.00 



.00 



.18 



25 
75 



50 
19 
12 
19 



25 
75 



75 
12 
0 
13 



3.20 



29 
71 

5:7 

29 
14 



29 
71 

57 
29 

14 



.00 



38 
62 

62 
23 

15 



23 
77 

62 
25 

23 



.40 



Method of Determining Open * 

Ed ucation Treatment 1 .11 

Observation or Questionnaire 69 25 

Open Space (Architecture) 12 56 

Judgment of Investisator(s) 19 19 

* 

Grade Level of Students 6 » 5 * 

Kindergarten to Grade 2 19 o 

Grade 3 to Grade 4 25 38 

Grade 5 to Grade 6 31 56 

Grade 7 or Above 25 0 



3.83 



1.47 



43 
14 
43 

14 
57 
29 



71 
29 
0 



29 
14 
57 



2.80 



69 
15 
16 



0 
46 
46 

8 



31- 
23 



23 
46 
31 
0 



4.40 



Ce*ree of Comparabilit y ,_of 
Open and Traditional Students 

* High 38 13 

Medium ° 50 62 

Lov ~ 12- * "25- 
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2.89 



14 
57 
29 



29 

;71 
0~ 



36 



2.44 



31 
23 
46 



31 
54 
15 



3.60 



Table 5, (continued) 



Characteristic of 
the Study 



SELF CONCEPT 1 CREATIVITY FAVORABLE ATTITUDE TOWARD SCHOOL 

X of Fisher's X of X of Fisher's X of X of Fisher's 

Small Exact Large Small Exact' Large Small E::act 

Fffect , Test Effect Effect Test Effect Effect Test 

Studies or X 2 Studies Studies or X 2 Studies Studies or X 



X of 
Large 
Effect 
Studies 



Country in Which the 
Study Was Implemented 

USA, - . 

Britain, Canada, or 
Australia 



9A 
6 



94 4 
6 



.00 



86 
1A 



86 
1A 



.00 



85 



15 



92 



.38 



D uration of Open Education 
Treatment 

Less than One Year 6 38 

One School Year 31 AA 

Between One and Two Years 19 6 

More than Two Years AA 12 



7.68 



0 
38 
12 
50 



13 
62 
13 
12 



3.30 



5A 
8 
15 
23 



23 
A6 
8' 
23 



5,50 



Quality of the Study 
Very Coed 
Moderately Good * 
Poor 



25 
62 
13 



13 
A7 
AO 



3.17 



1A 
72 
1A 



29 
A3 
28 



1.17 



2.60 



38 
23 
39 



23 
5A 
23 



Coiraitnenc of Investigator(s) 

Open Education 75 50 

Neutral 25 31 

Traditional Education 0 19 



3.91 



71 

29 



71 

29 



.00 



62 
38 
0 



5A 
31 
15 



2.18 



Note . * p < .10 
** p < .05 
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Tabic 6 



Conparison of. Selected Characteristics of the Studies Yielding Large Effects and Small Effects for the Student Outcbness 
Reading Achievement, Mathematics Achievement, and Language Achievement 



HEADING ACHIEVEMENT 



Characteristic of 
the Study 

Design -of~ the Study 

Correlational 

Experimental or Quasi- 
experimental 

Type of Publication 
Dissertation 
Journal Article 
Book, or Monograph 
ERIC Document 



X of. 
Large 
Effect 



X of 

Small 

Effect 



Studies' Studies 



Fisher's 
Exact 
Test 
or X 2 



12 
88 

47 
18 
6 
29 



12 

88 

59 
18 
12 
11 



.00 



1.84 



MATHEMATICS ACHIEVEMENT 

X of % of ' Fisher's 

Large Small Exact 

^Effect Effect Test 

Studies Studies or X' 



LANGUAGE ACHIEVEMENT 



93 



43 
36 
0 
21 



21 
89 



64 
7 
14 
15 



.38 



5.47 



X of 
Large 
Effect 
Studies 



X of 
Small 
Effect 
Stddies 



Fisher's 
Exact 
Test 
or X 2 



12 
88 

25 
38 
0 
37 



12 



88 



62 
0 
25 

13 * 



.00 



7.29 



Metho d of Determining Open 
Sduc:'- ion Treatment 

Observation or Questionnaire 30 

Open Space (Architecture) 35 

Judgment of Investigator (s) 35 

Grade Level of Students 

Kindergarten to Grade 2 24 

Grade 3 to Grade 4 **l 

Grade 5 to Grade 6 29 

Grade 7 or Above 6 



2.77 



.22 



1.03 



47 
41 
12 

35 
41 
24 
0 



1.51 



28 
36 
36 

14 

64 
22 



36 
36 
28 



29 
43 
28 



1.41 



38 
25 
37 



12 
50 
38 



63 
12 
25 

7.5 
38 
37 



.48 



Decr ee of Corpar ability of 
Open and '.'radlt ional Students 

High 

Med ium 

Lov * 



18 
53 
29 



53 
47 
0 



8.06 



** 



36 
28 
36 



29 
64 
7 



4.70 



0 

50 
50 . 



38 
50 
12 



4.80 



Table 6 (continued) 



READING ACHIEVEMENT 



MATHEMATICS ACHIEVEMENT 



LANo. 



ACHIEVEMENT 



.Characteristic of 
the Study 



X of % of Fisher's 

Large Snail Exact 

Effect Effect Test 

Studies Studies or X 2 



X of X of Fisher's 

Large Small Exact 

Effect Effect Test 

Studies Studies or x 2 



X of X of Fisher's 

Large Snail Exact 

Effect Effect Test 

Studies Studies or X 2 



* Country in Which the 
Study *as Implemented 

uSA v 

Britain* Canada, or 
. Australia^ 



76 
24 



70 
30 



.00 



71 

29 



79 
21 



.00 



100 
0 



75 
25 



(.23) 



Duration of Open Education 
Treatment 

less than One Year 

Cne School Year 

Between one/and Two Years 

More than Two Years 



18 
29 
18 
35 



29 
24 
18 
29 



.70 



22 
21 
21 
36 



29 
43 
7 
21 



2.64 



0 
38 
12 
50 



12 
63 
12 
13 



3.30 



Quality of thfe Study 
Very Good j 
Moderately Good 
Poor 



24 
35 
41 



19 
62 
19 



2.72 



.70 



2.31 



21 
50 
29 



23 
62 
15 



0 
62 
38 



25 
50 
25 



Comlt^ent ji Investigator (s) 
Open £<!•!&) t ion 
N'cijtral'J 

Traditional Education 



88 
12 

~0 



47 
47 
6 



6.73 



72 
21 
7 



57 
36 



.72 



88- 
12 
0 



38 
50 
12 



4.40 



Note. 



< .10 

< .05 



ERIC 
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of each other*, because some of the study characteristics covary. 4 Bui undue 
reliance should not be placed on either the .magnitude or statistical 
significance of these values in any event, ( Because the sample of •studies 
is small, the risk of a Yype II error is increased,) It is important to 
look for patterns in the ways the large effect °M small effect studies 
are distributed across categories, Regardless of statistical significance. 

Overall, there are few general statements, that can be made^ about dxff- 
erences in large effect and small effect studies on these study character- 
istics,-that are consistent across all six -student outcomes. One general 

finding that held for all student outcomes (except creativity) was that 

• * "* * 

the cdmmitment of the investigator was v more likelyf to be "favors open education 

for large effect than small effect studies, A,s'* ; as suggested earlier, , 

the tendency for a commitment to open education to he relat^ to large pfogram 

effects may be due to eit'her experimenter bias or experimenter thoroughness, 

.i e , the large effects* may be artifacts or they may reflect carefulness in 

implementing the open education program, 

Anoth'er general finding \that held for all the student outcomes 

(except for attitude toward school) was that large effect studies were more 

likely than small effect studies to have had a duration of the open education 

treatment of greater than two years. The small effect studies were more 

likely than -the large effect studies to have had a duration of treatment of 

less than one year. This finding that longer exposure to open education 

is related to larger effects is not unexpected. The failure of some open 

"education programs to produce positive effects may be due* in part, to ^ 

an inadequate duration of (weak) treatment. 

The three traditional achievement variables, rtading, mathematics, - 
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and language, often showed the same patterns of differences "between large effect 

and small effect studies, and in a direction that was often oppcsite to that 

• * 

for self concept, creativity, or attitude toward school, Fdr example, the 
method of determining the open education treatment was more likely to be 
observation or questionnaire for the small effect studies than the l^rge effect 
studies for reading, mathematics, and language. The opposite lield true for * 
the self concept and favorable attitude toward school outcomes.- Similarly, * 
the quality of the study was more likely to have been rated "poor" for 
large effect than for small effect studies for reading, mathematics, and 
language. For self concept and creativity, the small effect studies were 
more likely than the^large effect studies to have been rated "poor" in 
quality. 

With, the exception of self concept, the nine study characteristics 
also did not consistently distinguish between large effect and small effect 
studies within each student outcome, i.e., the different characteristics 
that were all indices of study quality and interpetability did not show the 
same direction of differences between large effect and small effect studies. 
Within any one student outcome, large effect studies might te more likely than 
small effect studies to have a high degree of comparability of students, 
yet also have been rated as poor in quality. For the student outcome self 
concept, however, all the indices pointed to the large effect studies 
as higher in quality, with greater treatment fidelity, and more validly 
able for causal inferences to be made about the reported results. 

Results of the Analysis of Open Education Features 

Tables 7 and 8 report the percentages of lajrge effect studies and the 
percentages of small effect studies for which each feature of open education 



Table 7 

Comparison of the Features of 
Self Concept, Creativity, and 



Open Education Found In the Studies Yielding Large Effects and*Small Effects' for the Student Outcomes: 
Favorable Attitude Toward School 



SELF CONCEPT 



»' CREATIVITY 



FAVORABLE ATTITUDE TOWARD SCHOOL 



feature of 
Open Education 



% of % of Fisher's 

Large Small Exact 

Effect Effect Test^ 

Studies Studies or X 2 



j% of % of Fishers 

^Large Snail Exact 

Effect Effect Test 

Studies Studies or 



% of % of Fisher's 

Large Small Exact 

Effect 0 Effect Test 

Studies Studies or X 2 



Open Srace 
Yes 
No 

\ 

Materials to Manipulate 
Yes 
No 



87 
13 



100 
0 



81 
19 



27 
73 



.006 



14.350*** 



71 

29 



\ 80 
20 



71 

29 



50 
50 



(1.000) 



(.348) 



54 
46 



82 
18 



85 
15 



56 
44 



(.101)** 



(.217) 



>!u lti-nr.c Grouping 
of Students 

Yes 

No 



81 
19 



75 
25 



.000 



83 

\ 

17. 



33 
67 



(.121)* 



46 
54 



46 
54 



(.621) 



Individualized 
Instruct Ion 

Yes 

No 



100 
0 



44 
56 



9.894*** 



83 ' 
17 \ 



67 
33 



(.500) 



80 
20 



75 
25 



(.594) 



Role of Child 
in Looming 

Yes * 

* No 



100 
0 



50 
50 



8.167*** 



86 
14 



67 
33 



(.437) 



85 
15 



69 
31 



(.322) 



Teas Teaching 
Yes 
No 



67 
33 



69 
31 



.068 



40 
60 



57 
43 



(.500) 



36 
64 



62 
38 



(.207) 



Diagnostic Evaluation 
Yes 




100 
0 



46 
54 



(.002)*** 



42 



100* 
0 



50 
50 



(.091)** 



73 
27 



50 
50 



(.297) 



Tabic 7 (continued) 



Feature of 
Open Education 



SELF CONCEPT 

Z of Z of Fisher's 

Large Snail Exact 

Effect Effect Test 

Studies Studies or X 2 



Z of 
Large 
Effect 
Studies 



CREATIVITY FAVORABLE ATTITUDE TOWARD SCHOOL 

Z of Fisher's Z of Z of Fisher's 

Small Exact Large Small Exact 

Effect Test v Effect Effect Test 

Studies or X 2 Studies Studies or X 2 



Extensiveness of Open 
Education Treatment 

High 

Medium 

Low 



75 
25 
0 



25 
13 
62 



14.670*** 



43 
29 
28 



29 
14 
57 



1.200 



46 
8 
46 



23 
15 
62 



1.620 



Specif Icty of Theory 
of Open Education 

High 

Medium 

Kow 



19 
62 
19 



37 
44 
19 



1.530 



29 
43 
28 



43 
43 
14 



.530 



31 
54 
15 



54 
31 
15 



1.640 



E:rphasls of Theory 
of Open Education 

Open Space 

Teaching 

Both 



0 

56 
44 



50 

38 
12 



11.380*** 



14 

57 
29 



29 
71 
0 



2.440 



15 
77 
8 



31 
54 
15 



1.530 



Mote. Studies for which there was no information on a particular feature and for which an informed judgment could not be irade 
were excluded from the analysis for that feature. Median percentage of cases excluded for all features, for all 
.student outcomes, was 7Z of total sample. ^ 

Fisher's Exact Probability was computed for 2 X 2 analyses when N < 30; these probabilities are in parentheses. Chl-square 
was computed when N > 30 studies and for all the 2X3 analyses; these values are not in parentheses. * 

* p < .15 ** p < .10 *** p < .01 
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Table 8 ■ 

Comparison of the Features of Open Education Found in the Studies Yielding Large Effects and Small Effects for the Student Outcomes: 
Reading Achievement, Mathematics Achievement, and Language Achievement 



Feature of 
Open Education 



READING ACHIEVEMENT 



■ MATHEMATICS ACHIEVEMENT 



% of 
Large 
Effect 
Studies 



7. of 
Small 
Effect 
Studies 



Fisher's 
Exact 
Test 
or X 2 



% of 
Large 
Effect 
'Studies 



% of 
Small 
Effect 
Studies 



Fisher's 
Exact 
Test 
or x 2 



LANGUAGE ACHIEVEMENT 

% of 7. of Fisher's 

Large Small Exact 

Effect Effect Test 

Studies Studies or X 2 



jOpcn Space 
Yes 
No 



75 
25 



94 
6 



1.092 



71 

29 



79 
21 



(.500) 



75 
25 



88 
12 



(.500) 



Materials to Manipulate 

Yes 77 50 

No 23 50 



(.147)* 



67 
33 



54 
46 



(.433) 



83 
17 



67 
33 



(.500) 



M ulti-age Grouping 
of Students 



:es 
No 



71 

29 



65 
35 



.000 



57 
43 



79 
21 



(.210) 



57 
43 



75 
25 



(.427) 



Indiv idualized 

Instruction 



Yes 
No 



85 
15 



77 
23 



(.469) 



83 
17 



86 
14 



(.641) 



86 100 
14 0 



(.467) 



Role of Child 
in Learnin g 

Yos 

No 



75 
25 



75 
25 



.167 



71 

29 



69 
31 



(.615) 



86 
14 



88 
12 



(.733) 



Team Teaching 
Yes 
No 



73 
27 



50 
50 



.929 



62 
38 



54 
46 



(.500) 



71 

29 



38 
62 



(.214) 



Diagnostic Evaluation 



Yes 
N'o 



82 
13 



77 
23 



(.585) 



44 



80 
20 



83 
17 



(.632) 



.86 
14 



83 
17 



(.731) 



JTable 



8 (continued) 



Feature of 
Open Education 



READING ACHIEVEMENT 



X of 
Large 
Effect 
Studies 



% of 
Small 
Effect 
Studies 



Fisher's 
Exact 
Test 
or X 2 



MATHEMATICS ACHIEVEMENT 



LANGUAGE ACHIEVEMENT 



% of 
Large 
Effect 
Studies 



% of 
Small 
Effect 
Studies 



Fisher's 
Exact 
Test 
or X 2 



%.of 
Large 
Effect 
Studies 



% of 
Small 
Effect 
Studies 



Fisher's 
Exact 
Test 
or X 2 



Extenpiveness of Open 
Educa tion Treatment 

High 

Medium 

Low 



24 
47 
29 



24 
31 
35 



.160 



21 
50 
29 



36 
36 
28 



.830 



50 
25 
25 



38 
50 
12 



1.140 



Specific i ty of Theory 
of Open Educa tion 

High 

Medium ' 
Lew 



30 
35 
35 



18 

76 
6 



6.650** 



36 
36 
28 



14 
64 
22 



2.570 



38 
25 
37 



0 
75 
25 



5.200* 



Emphasis of Theory 
of Open Education 

Open Space 

Teaching 

Both ' 



30 
35 
35 



24 
53 
23 



1.110 



29 
43 
28 



14 
64 
22 



1.410 



25 
38 
37 



0 
88 
12 



4.600 



N'ote. 



Studies for which there was no information on a particular feature and for which an informed judgment could not be made 
were excluded from the analysis for that feature. Median percentage of cases excluded was 7Z. 

Fisher's Exact Probability was computed for 2 X 2 analyses when N < 30; these probabilities are in parentheses. Chi-equare 
was computed when N > 30 studies and for all 2X3 analyses; these values are not in parentheses. 



< .15 



.10 



*** p < .01 
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vas present or absent. The values of chi-square or Fisher's Exact Test 
are also given. Table 7 reports this information for the student outcomes: 
self concept, creativity, and favorable attitude toward school. Table 8 
reports this information for the student' outcomes: reading achievement, 
mathematics achievement, and language achievement. 

Table 9 reports the mean number of features of open education 
found in the large effect and small effect studies which measured each student 
outcome variable, T-tests for differences between the means are also reported- 
Table 1G summarizes tb*> direction of the differences between large effect and small eff 

stadies on each feature of open education, regardless of the statistical 
significance of the difference. 

* As was true of study characteristics, there are few features of^open 
education that^distinguish large effect from small effect studies for all 
six student outcome variables. The student outcome variables did seem to 
cluster into two groups in terms, of consistency in directions of differences 
reported for each open education feature. Self concept, creativity, and 
favorable attitude toward school formed one cluster, while language achieve- 
ment and mathematics achievement formed another cluster. Reading achievement 
seemed to alternate between the self concept cluster and the achievement 
cluster in the direction of differences between^arge effect and sinall 
effect studies that it displayed. The two clusters often showed opposite- 
directions of differences. t 

The mean number of features of open education (reported in Table 9) 
v;hich were imnlsmented showed little differences between large effect and small 
effect studies, except 'for self concept and creativity. For self concept, 
large effect studies contained an average of two more features than small 
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Table 9 

Differences between Large Effect and Small Effect Studies in the Kean Number 
of Features of Open Education which Were Implemented 



Large Effect Studies Small Effect Studies 

Student Outcomes M SD n M SD n t 

Self Concept 6.12 ,81 16 3,88 1.78 16 4.60 

•Creativity 4.71 1.80 7 ~- 3.71 1.38 *7. 1.17 
Favorable Attitude 

Toward School 4.15 1.68" 13\ 3.92 1.75 13 0.34 

Reading Achievement 4.47 1.46 17 4.53 1.46 17 -0.12 
Mathematics 

Achievement 4.43 1.02 14 4.71 1.77 14 ' -0.52 
Language 

Achievement 4.75 1.75 8 5.00 1.69 8 -0.29 



Mote . Maximum number of features is 7, 
* p < .01 
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.Table 10 

Susciary of the Direction of "Differences between Large Effect and Small Effect Studies on Features of Open Educatibn 
Feature of Favorable Attitude 

'Open Education Self Concept Creativity Toward School Reading Mathematics Language 



Open Space 

r 

Materials to 
Manipulate 

tfulti-age Ciouping 
of Students 

i 

' Individualized 
Instruction 

Pole of Child 
in Learning . 

Te*"t. Teaching 

Diagnostic 
Evaluation 



L > S No Diff. S > L**_ S >__L S„> X S->-L— 

L > S*** ™ L > S L>S L > S* L>S L>S 

L > S L > S** No Diff. L>S S>L S>L 

L > S*** L>S L>S L>S S > L S > L 

U> S*** L > S L > S No Diff. "L > S - S > L 

S >L S>L S>L L>S L > S L>S 

L > S*** L > S** L>S L>S-S>L L>S 



oT^kSr L > L > S K. Diff. " No Diff. Ho Diff. No Diff. 



AAA 



Extonsiveross of 
Open Treatment 

^ S > L S > L S > L S > L L > S L > S 

L >S L>S S>L L ^ S L > S, S>L 

L > s * l > S L > S ~No Diff.* S > L L > S 



Medium 
High 



Spccificty of Theory . ' aa * 

^of Op«n education 

Uo Diff. ' L > S No Diff, L>S L > S« L>S 

L > S t No Diff. L > S , * S>L S>L S>L 

S >L S > L * S>L L>S L>S L>S 



Low 

v ed iuin 
Hifh 



Emphasis pf Theory A 
of Ooen Education *** 



Open Space 

leaching 

Both 



S >L S > L ' S>L L>S L>S L>S 

L>S S>L L>S S > L ' S > L S>L 

L > s L>S S>L L>S L > S L>S 



Vote. , V means more' likely to include that feature of open education. "L" - large effect studies. "S" - small 
" O feet studies. 

Ha^Bap<,l5 **p<.l0 *** p << .01 , 48 
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effect studies, ^ The t-ratic for this difference between means was significant. 
The large effect studies for self concept also included between one and 
„_ t Jtwo_inore- -features of open education than the* large effect studies for the 
other student outcome variables, r 

For creativity, large effect studies contained an average of one rsore 
feajzure than small effect studies, c 1 , 0 

The direction of the difference between mean number of features for 
large effect and small effect studies was negative for the three achievement.' 
outcomes, suggesting that while more features are related to positive 
effects for self concept, creativity, and favorable attitude toward school, 
more features are related to negative effects for reading, mathematics, 
and language achievement. This trend is also shown for other features of 
open education, as reported in Table 10, 

The one general "finding that held across all six student outcomes v/as, 
that large effect studies were more likely to include manipulative naterials 
as a program feature than small effect studies. Diagnostic evaluation 
was another feature that was mere likely to be included in large effect than 
small effect studies for most of the six student outcome^. 

Individualized instruction and team- teaching are two features that v 
demonstrated reverse directions of differences for the self concept, creativity, 
attitude cluster and the language and math achievement cluster. Individualized 

e 

instruction was irore likely to be found in large effect than small effect 
studies for the outcomes self concept, 'creativity, and attitude toward school, 
while individualized instruction was. more likely to be found in small effect 
studies than large effect studies fot the outcomes, language and mathematics 
achievement: Team teaching was more likely to he found in the **mall effect < 
than large effect studies for self concept, creativity , and attitude toward 

* r 

school while the opposite was true for reading, mathematics, and language. 

49. • ■ 
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Similarly, ratings of the extensiveness of the treatment as low, specificity 
of the theory of open education as high, and 'emphasis of the theory of open education 
as open space, showed opposite directions of differences for the self concept, 
•creativity, attitude toward school outcomes and the reading, nathematics , and 
language. achievement outcomes. For self concept, creativity, and attitude toward 
school, small effect studies were more likely than large effect studies 
to have been of low extensiveness, to have been of high specificity of the 
theory, of open education on .which the program was based, and to have emphasized 
open space rather than teaching practices. The opposite was true. for the 
achievement outcomes'. 

The- consistency within each outcome^ variable was highest for self 
concept, creativity, and reading. That is, the direction of the differences 
between large effect and small effect studies was similar across the different 
features of open education. For language arid mathematics achievement, the 

A. T 

^direction of differences favored large effect studies for about half of the 
features and favored small effect -studies for half of the features. 

The profile t'hat emerges of the effective open education program 
depends, of course, on the des.red student outcomes. For outcomes such as 
self concept, creativity, and favorable "attitude toward school, the effective 
open education program is characterized as including most of the seven 
features identified in this stu'y, as extensive in both breadth (number ot 
features) and depth (extensiveness rating). This effective program also emphasizes 
teaching practices or teaching practices and open space, rather than open 
space alone. The effective program is not based on a specific theoretical 
model of open education, but draws on several different ideas jand models. 

For outcome such as mathematics and language achievement ,^more is not - 
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better. t% Large effects for these outcomes are related to less extensiveness 
and fewer number of features of open education. Emphasis on open .space rather 

» 

than teaching practices is called for. 

Implications for Educational Research and Practice 

f The call for more studies of open education may seem ill-advised, 
given the burgeoning body of open education studies'. But what are needed 
are studies to systematically test the causal efficacy of various configurations 
of the open education features that have been "shown in this study to be related 
to large program effects. Studies of this sort can determine the relative 
contribution of each feature to overall program effects. This information 
can\then be used to identify a set of necessary and sufficient features 
of effective open education programs. 

More generally, this study has illustrated a strategy for educational 

\ . 

researchers" to use when faced with a body of literature that shows a high 
degree of inconsistency in results. That is, an effect size for each 
•study can be computed. The two extreme groups of studies on effect size 
can be compared and contrasted on both study design characteristics and 
dimensions or features of the treatment variable in question. . Such an 
approach works best when the effect sizes across studies are not homo- 
geneous, i.e., are highly variable. Hedges' (1980) statistical test 
of homogeneity of effect sizes can be used to establish this. 

Recommendations for educational practice or the design of open 
education programs are tentative at best, until the causal efficacy of 
some of the fpftture^ identified in this study is established. But in general, 
the educator who is primarily interested in self concept, creativity, and 
attitude toward school and learning as desired student outcomes, is advised 



to 'include as many of the ^features of open education identified in this^ 
study (open space, manipulative materials, multi-age grouping of students, 
individualized^ instruction, role of the child in learning, team teaching, 
and diagnostic evaluation) as possible. 
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Table A 

Classification of the 7Z :tudi-.r> as Large Effect or Sna*:. T-ff 3Ct 



Author(s) 



Self 



Attitude 
Toward 



Concept ' Creativity School 
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Reading Math Language 



Abelson, Zigler, 
and DeBlasi (1974) 

Allen (1974) 

Arlin and Palm 
(1974) 

Bell, Switzer, 
and ^Zipursky % 
(1974) 

Bell, Zipufsky, 
and Switzer 
(1976) 

Bennett (1976) 

Broward County 
School Board 
(1972) 

Brown (1973) 

Burychett (1972) 

Burnham (1971) 

Butson (1975) 

Case (1970) 

Cockerham and 
B lev ins (1976) 

Daniels (1974) 

Day (1974) 

Dornseif (1975) 

Dugan (1976) 

Earnshaw ,(1972) 

Form™ (1975) 



L 
S 

L 
S 

L 
S 



L 

S 



S 
L 



L 
L 



L 
L 
S 



L 
L 



L 
S 



L 
L 
L 

.S 
S 
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Author(s) 



Self 
Concept 



Creativity 



Attitude 

Toward 

School 



Reading Math Language 



Fox (1975) 

Franks, Marolla, 

and Dillon (1974) L 

Franks (1977) L 

Glinsky (1973) 

Godde (1972) 

Grapko (1972) S 

Grogan (1976) 

Groobman, Forward, 9 
and Peterson (1976) S 

Hill (1973) 

Hopke (1974) L 
Horwit* (1976) 
Hudson (1973) 
Jensen (1976) 
Klein (19>5) 

Koskoff (1^973) L 

Kourilsky and 
Baker (1975) 

Leroy (1973) 

McBride (1975) 

McCorkle (1974) L 

Meadow (1973) S 

Mealor, Perkins, 
and Reeves (1975) 




L 
L 



S 
L 
S 
L 



S 
L 



L 

S 



L 
L 



L 

S 
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Table A (continued) 
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Author (s) 



Attitude 
Self Toward 
Concept Creativity School 



Reading Math Language 



\ 



Mills (1975) 

Moore (1974) 

Morris (1977) 

Nixon (1973) 

Nogrady (1975) 

O'Neill (1974) 

Ramey and 
Piper 41-974) 

Reeder (1975) . 

Reynolds (1974) 

Riley (1976) 

Robinson (1974) 

Rothschild (1976) 

Rozar (1976) 

Sackecc (1971) 

Scheiner (1969) 

Scheirer (1972) 

Sewell (1975) 

Shapiro (1971) 

Shopland (1975) 

S cowers (1974) 

Travers (1974) 

Trotta (1973) 

Ward and Barcher 
(1975) 



S 
S 



s 

L 

S 



S 
L 
S 



S 
L 



L 

S 



S 

S S 
L L 



S 
L 



S 
L 
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jp Table A (continued) 



i Self 
Author (s) J Concept 

Weiss (1971) 

White (1973) S 

Wright (1974) S 

York County Board 
of Education (1970) 

Reynolds (1974) L 

Pine (1977) L 

Bell, Abrahamson, 
and Growse (1977) 

Seidner (1978) S 

Angus, Beck, Hill, 

and McAtee (1979) L 



Attitude 
Toward 

Creativity School 
L 

S 

S S 
S 



Reading Math Language 

S S 

L 
S 
L 

S S 

L L L 

S S S 



Note . "L M == large effect study 
"S" = small effect study 

"-" indicates that either the study was a medium effect 
study or did not report any results for that student 
outcome. 
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Table B . 53 



a 

Measures Used for Student Outcomes 



b 

Self Concept (n=32) 



(1) Attitude Questionnaire, Developed by Investigator, Jensen(1976) , 
Personal Satisfaction Scale 

(1) California Test of Personality, Sense of Personal Worth Scale 

(9) Coopersmith Self Esteem -Inventory , Academic Situation Scale and 
Total Score 

(2) Gordon's How I See Myself, Academic Adequacy Scale, Interpersonal 
^ . . Adequacy Scale, and Total Self Concept 

(2) Marolla Personal Competence, Inner Self Esteem Score 

(7) Piers-Harris Children's Self Concept Scale, Self Approval Scale 
' and Total Score 

(2) Self Concept and Motivation Inventory (SCAMIN), Self Adequacy 
Scale and Total Self Concept 

(1) Self Concept as a Learner Scale, Total Score 

'(1) Self Concept Measure, Rookey and Valdes (1972) , Total Score 

(1) Self Esteem, Developed by Investigators, Groobman et al.(1976), 
Total Score 

(1) Self Social Symbols Task and Children's Self Social Constructs 
Tests, Ziller et al.(1971;), Esteem Scale 

(1) Student Self Image developed by Investigators, Abelson et al.(197A), 
Total Score 

(1) The Institute of Child Study Security Test, "The Story of Jimmy, " 
Security Scale 

(1) Thomas Self Concept Values Test, Total Score 

(1) Twenty Statements Test, Kuhn and McPartland(1954 ,1960) , Scored 
for Self Derogation 
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Table B (continued) 



5A 



Creativity (n=14) 

(1) Group Test of Creativity, Metfessel et al. (1972), Total Score 

(1) "How I Would Change My School" Essay, Scored for Creativity 

(1) Pennsylvania Assessment of Creative Tendency, Rookey (1971) , 
Total Score 

(1) Test of Solving Puzzles, Kaya (1965), Divergent Thinking Score 
(8) Torrance Testes of Creative Thinking, Total Scoxe on All Scales 

(2) Wallach-Kogan Creativity Measure, Total Uniqueness and Total 

Fluency Scores 

Attitude Toward School (n=26) 

(1) Attitude Questionnaire, Developed by Investigator Based on 
Schulman (1972), Environmental Satisfaction ^ 

(1) Attitude to Schooling, Developed by Investigator, Day (1974), 
Total Score 

(1) Attitude Toward Learning Processes, Arlin and Hills (1975), 
Total Score 

(1) Attitude Toward School Inventory, Developed by Investigator, 

Scheiner (1969), Total Score 

(2) Attitude Toward School Questionnaire, Barker Lunn (1970), Total 

on All Scales 

(3) Attitude Toward School Questionnaire, Klein and Strickland (1970), 

Total Score 

(1) Classroom Self Perception Report, Developed by Investigator, 
Leroy (1973) 

(1) ETC Attitude Toward School, Frieze (1972), Total Score 

(2) Faces Test, School-Learning Scale 

(1) Favorability Toward Schoo 1 , Paskal and Weiss (1971), Total Score 
(1) Fitt Study of Attitudes, Scored for Attitude Toward School 
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Attitude Toward School (continued) 

(1) Gordon's How I See Myself, Teacher-School Favorability Scale 

(5) School Sentiment Index, Total Scoreand General Notions About 
School Scale 

(1) Semantic' Differential, Concept Rated: "School" 

(1) Student Attitude Toward School Measure (SATSM) , General Scale 

(1) Student Attitude Toward School, and Learning, Developed by Investigators, 1 
Groobman et al. (1976), Total Score 

(1) Student Opinion Poll II, Getzels and Jackson (1962), School Scale 

(1) Test to Measure a Child's Attitude, Tenenbaum (1940) , Attitude 
Toward School Scale , 



Reading (n=34) 

(1) A.C.E.R. Primary Reading Survey Test, Reading Comprehension 

(2) California Achievement Test, Reading Total 

(2) Canadian Tests of Basic Skills, Reading Comprehension 
(1) Cooperative Primary Tests, Reading 

7 

(1) Edinburgh Reading Test, Total Score 

(4) Gates-MacGintie Reading Tests, Reading Comprehension 

i- 

(5) Iowa Tests of Basic Skills, Reading Comprehension 

(7) Metropolitan' Achievement Test, Reading Comprehension 
(1) Peabody Individual Achievement Tests, Reading • 
(1) SCAT/STEP, Reading Total 

(8) Stanford Achievement Test, Paragraph Meaning and Reading Total 
(1) Wide Range Achievement Test, Reading ^ 
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Table B (continued) 



Mathematics (n=28) t 

(1) A.cTe.R, Mathematics Test, Money AM5 

(2) California Achievement Test, Mathematics Total 

(3) Canadian Tests of Basic Skills, Arithmetic Total 
(1) Cooperative Primary Tests, Number Concepts 

(6) Iowa Tests of Basic Skills, Arithmetic Total 

(4) Metropolitan Achievement Test, Mathematics Total ' A 
(1) N.F.E.R. Mathematics Attainment, Mathematics Total 

(1) Peabody Individual Achievement Tests, Mathematics 

(7) Stanford Achievement Test, Mathematics Total 

(2) STEP, Mathematics 

Language (n=16) 

(1) California Achievement Test, Language Total 

(1) Canadian Tests of B sic Skills, Language Skills Total 
(4) Iowa Tests of Basic Skills, Language Skills Total 

(2) Metropolitan Achievement Test, Language Total 

(1) N.F.E.R* English Progress Test, Total Score 

(2) SCAT/STEP, English Expression 

(4) Stanford Achievement Test, Language Total 

' (1) Written Expression I and II, * Developed by Investigators, Angi 
et al.(1979), Total Score 

/ *> 

Note* a Number in parentheses is the number of large and small 
effect studies in which that measure was used. 

b Total number of large and small effect studies 

0 

» I 
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